我们研究了从记录的匪徒反馈中进行额外学习的增强合奏模型。为了实现这一目标，我们提出了一种新的增强算法，该算法直接优化了对政策预期奖励的估计。我们分析了该算法，并证明，只要满足“弱”的学习条件，每轮增强的经验风险会随着每一轮增强而降低（可能是指数迅速）。我们进一步展示了基础学习者如何减少标准监督学习问题。实验表明，我们的算法可以胜过仅在观察到的奖励上回归的深层外部学习和方法，从而证明了增强和选择正确的学习目标的好处。

Search and Rescue (SAR) missions in remote environments often employ autonomous multi-robot systems that learn, plan, and execute a combination of local single-robot control actions, group primitives, and global mission-oriented coordination and collaboration. Often, SAR coordination strategies are manually designed by human experts who can remotely control the multi-robot system and enable semi-autonomous operations. However, in remote environments where connectivity is limited and human intervention is often not possible, decentralized collaboration strategies are needed for fully-autonomous operations. Nevertheless, decentralized coordination may be ineffective in adversarial environments due to sensor noise, actuation faults, or manipulation of inter-agent communication data. In this paper, we propose an algorithmic approach based on adversarial multi-agent reinforcement learning (MARL) that allows robots to efficiently coordinate their strategies in the presence of adversarial inter-agent communications. In our setup, the objective of the multi-robot team is to discover targets strategically in an obstacle-strewn geographical area by minimizing the average time needed to find the targets. It is assumed that the robots have no prior knowledge of the target locations, and they can interact with only a subset of neighboring robots at any time. Based on the centralized training with decentralized execution (CTDE) paradigm in MARL, we utilize a hierarchical meta-learning framework to learn dynamic team-coordination modalities and discover emergent team behavior under complex cooperative-competitive scenarios. The effectiveness of our approach is demonstrated on a collection of prototype grid-world environments with different specifications of benign and adversarial agents, target locations, and agent rewards.

By transferring knowledge from large, diverse, task-agnostic datasets, modern machine learning models can solve specific downstream tasks either zero-shot or with small task-specific datasets to a high level of performance. While this capability has been demonstrated in other fields such as computer vision, natural language processing or speech recognition, it remains to be shown in robotics, where the generalization capabilities of the models are particularly critical due to the difficulty of collecting real-world robotic data. We argue that one of the keys to the success of such general robotic models lies with open-ended task-agnostic training, combined with high-capacity architectures that can absorb all of the diverse, robotic data. In this paper, we present a model class, dubbed Robotics Transformer, that exhibits promising scalable model properties. We verify our conclusions in a study of different model classes and their ability to generalize as a function of the data size, model size, and data diversity based on a large-scale data collection on real robots performing real-world tasks. The project's website and videos can be found at robotics-transformer.github.io

Decentralized learning with private data is a central problem in machine learning. We propose a novel distillation-based decentralized learning technique that allows multiple agents with private non-iid data to learn from each other, without having to share their data, weights or weight updates. Our approach is communication efficient, utilizes an unlabeled public dataset and uses multiple auxiliary heads for each client, greatly improving training efficiency in the case of heterogeneous data. This approach allows individual models to preserve and enhance performance on their private tasks while also dramatically improving their performance on the global aggregated data distribution. We study the effects of data and model architecture heterogeneity and the impact of the underlying communication graph topology on learning efficiency and show that our agents can significantly improve their performance compared to learning in isolation.

This work addresses cross-view camera pose estimation, i.e., determining the 3-DoF camera pose of a given ground-level image w.r.t. an aerial image of the local area. We propose SliceMatch, which consists of ground and aerial feature extractors, feature aggregators, and a pose predictor. The feature extractors extract dense features from the ground and aerial images. Given a set of candidate camera poses, the feature aggregators construct a single ground descriptor and a set of rotational equivariant pose-dependent aerial descriptors. Notably, our novel aerial feature aggregator has a cross-view attention module for ground-view guided aerial feature selection, and utilizes the geometric projection of the ground camera's viewing frustum on the aerial image to pool features. The efficient construction of aerial descriptors is achieved by using precomputed masks and by re-assembling the aerial descriptors for rotated poses. SliceMatch is trained using contrastive learning and pose estimation is formulated as a similarity comparison between the ground descriptor and the aerial descriptors. SliceMatch outperforms the state-of-the-art by 19% and 62% in median localization error on the VIGOR and KITTI datasets, with 3x FPS of the fastest baseline.

This paper is a technical overview of DeepMind and Google's recent work on reinforcement learning for controlling commercial cooling systems. Building on expertise that began with cooling Google's data centers more efficiently, we recently conducted live experiments on two real-world facilities in partnership with Trane Technologies, a building management system provider. These live experiments had a variety of challenges in areas such as evaluation, learning from offline data, and constraint satisfaction. Our paper describes these challenges in the hope that awareness of them will benefit future applied RL work. We also describe the way we adapted our RL system to deal with these challenges, resulting in energy savings of approximately 9% and 13% respectively at the two live experiment sites.

Grammatical Error Correction (GEC) is the task of automatically detecting and correcting errors in text. The task not only includes the correction of grammatical errors, such as missing prepositions and mismatched subject-verb agreement, but also orthographic and semantic errors, such as misspellings and word choice errors respectively. The field has seen significant progress in the last decade, motivated in part by a series of five shared tasks, which drove the development of rule-based methods, statistical classifiers, statistical machine translation, and finally neural machine translation systems which represent the current dominant state of the art. In this survey paper, we condense the field into a single article and first outline some of the linguistic challenges of the task, introduce the most popular datasets that are available to researchers (for both English and other languages), and summarise the various methods and techniques that have been developed with a particular focus on artificial error generation. We next describe the many different approaches to evaluation as well as concerns surrounding metric reliability, especially in relation to subjective human judgements, before concluding with an overview of recent progress and suggestions for future work and remaining challenges. We hope that this survey will serve as comprehensive resource for researchers who are new to the field or who want to be kept apprised of recent developments.

The computational complexity of the self-attention mechanism in Transformer models significantly limits their ability to generalize over long temporal durations. Memory-augmentation, or the explicit storing of past information in external memory for subsequent predictions, has become a constructive avenue for mitigating this limitation. We argue that memory-augmented Transformers can benefit substantially from considering insights from the memory literature in humans. We detail an approach for integrating evidence from the human memory system through the specification of cross-domain linking hypotheses. We then provide an empirical demonstration to evaluate the use of surprisal as a linking hypothesis, and further identify the limitations of this approach to inform future research.

通过将从地面视图摄像头拍摄到从卫星或飞机上拍摄的架空图像的图像，通过将代理定位在搜索区域内，将代理定位在搜索区域内，将代理定位在搜索区域中。尽管地面图像和架空图像之间的观点差异使得跨视图地理定位具有挑战性，但假设地面代理可以使用全景相机，则取得了重大进展。例如，我们先前的工作（WAG）引入了搜索区域离散化，训练损失和粒子过滤器加权的变化，从而实现了城市规模的全景跨视图地理定位。但是，由于其复杂性和成本，全景相机并未在现有机器人平台中广泛使用。非Panoramic跨视图地理定位更适用于机器人技术，但也更具挑战性。本文介绍了受限的FOV广泛地理定位（Rewag），这是一种跨视图地理定位方法，通过创建姿势吸引的嵌入并提供将粒子姿势纳入暹罗网络，将其概括为与标准的非填充地面摄像机一起使用，以供与标准的非卧型地面摄像机一起使用。 Rewag是一种神经网络和粒子滤波器系统，能够在GPS下的环境中全球定位移动代理，仅具有探测仪和90度FOV摄像机，其本地化精度与使用全景相机实现并提高本地化精度相似的定位精度与基线视觉变压器（VIT）方法相比，100倍。一个视频亮点，该视频亮点在https://youtu.be/u_obqrt8qce上展示了几十公里的测试路径上的收敛。

从单个放射学图像中学到的功能无法提供有关随着时间的流逝可能发生的病变以及多少变化的信息。从重复图像计算出的时间相关特征可以捕获这些变化，并通过其时间行为来识别恶性病变。但是，纵向医学成像提出了数据获取的稀疏，不规则时间间隔的独特挑战。虽然自我注意事项已被证明是时间序列和自然图像的一种多功能，有效的学习机制，但尚未探索其在稀疏，不规则采样的空​​间特征之间解释时间距离的潜力。在这项工作中，我们通过使用（1）连续时间的矢量嵌入和（2）时间强调自我注意力的权重来提出两种解释时间距离视觉变压器（VIT）。这两种算法是根据合成肺结节的良性与恶性肺癌区分和肺筛查计算机断层扫描研究（NLST）评估的。与标准VIT相比，评估合成结节的时间段VIT的实验表明，在对不规则采样的纵向图像进行分类方面有了基本改进。在从NLST筛选胸部CTS的交叉验证中，我们的方法（分别为0.785和0.786 AUC）显着超过了横截面方法（0.734 AUC）（0.734 AUC），并匹配领先的纵向医学成像算法（0.779 AUC）在良好的良性上的判别性能与恶性分类。这项工作代表了第一个基于自我注意的框架，用于对纵向医学图像进行分类。我们的代码可从https://github.com/tom1193/time-distance-transformer获得。